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The concavity of Renyi entropy power 

Giuseppe Savare and Giuseppe Toscani 



(N- 

o: 

(N- 
< 

in 



> 

in 
m 
o 

od 
O 



X 



Abstract — We associate to the p-th Renyi entropy a definition 
of entropy power, which is the natural extension of Shannon's 
entropy power and exhibits a nice behaviour along solutions 
to the p-nonlinear heat equation in R n . We show that the 
Renyi entropy power of general probability densities solving such 
equations is always a concave function of time, whereas it has 
a linear behaviour in correspondence to the Barenblatt source- 
type solutions. This result extends Costa's concavity inequality 
for Shannon's entropy power to Renyi entropies. 

Index Terms — Entropy, information measure, information the- 
ory, Renyi entropy, nonlinear heat equation. 

I. Introduction 

The p-th Renyi entropy of a random variable X with 
density / in R™ is defined by (see, e.g. Cover and Thomas 
and Gardner 02)) 

J{ p pO = H p (/) := -J- log f p(x) dx, (1) 

for < p < +oo, p 7^ 1. 

Whenever p > 1 — 2/n, we consider the positive coefficients 



H :=2 + n(p-l), 



A 4 2 

- = - + (p ■ 

n n 



1) 



(2) 



and we associate to the p-th Renyi entropy the entropy power 
(that we call p-th Renyi entropy power in the following) given 
by 

M p (X)=N p (f):=exp(vX p {X)). (3) 

The Renyi entropy for p = 1 is defined as the limit of l K p {X) 
as p — » 1. It follows directly from definition ([T]i that 



Mi(X) = lim "KJX) = MX) 



f(x) log/(x) dx. 



Therefore, the Shannon's entropy can be identified with the 
Renyi entropy of index p = 1. In this case, the proposed 
Renyi entropy power of index p = 1, given by (0, coincides 
with Shannon's entropy power 

N(X) = N(/):=exp/-?{(X)l. (4) 



In 1985 Costa proved that, if u(-,t), t > 0, are probability 
densities solving the heat equation 

d 



Of 



= Ai 



(5) 



posed in the whole space 
dt 2 



then 

N(<it(-,i)) < 0. 
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Inequality ((6]) is referred to as the concavity of entropy power 
theorem (see Riul ifTTll for an exhaustive list of references). 
The original proof of Costa has been simplified years later by 
Dembo iflOl . ifTTI with an argument based on the Blachman- 
Stam inequality |0. Next, a direct proof of (0 in a strength- 
ened form, with an exact error term, has been obtained by 
Villani 11221 . The proof in l22l highlights a strong connection 
between the concavity of entropy power and some identities 
of Bakry and Emery |2|, established through the so-called I^ 
calculus as part of their famous work on logarithmic Sobolev 
inequalities and hyper-contractive diffusions. These connec- 
tions, together with various consequences of the concavity of 
entropy power theorem, have been recently discussed in |fl9l . 

In this paper we show that the concavity of entropy power 
is a property which is not restricted to Shannon entropy power 
in connection with the heat equation ©, but it holds for 
the p-th Renyi entropy power (O, if we put it in connection 
with the solution to the nonlinear heat equation 

^-u = AuP, (7) 
at 

posed in the whole space R n . The precise result is the 
following. 

Theorem 1: Let p > 1 — 2/n and let u(-,t) be probability 
densities in R" solving (0 for t > 0. Then the p-th Renyi 
entropy power defined in (O satisfies 

d 2 

-^N p (ii(-,i)) < i.e. t ^ N p («(-,*)) is concave. (8) 

Like in the Shannon's case, inequalities (O lied to sharp 
isoperimetric inequalities. It is remarkable that the range of 
the parameter p for which we can introduce the Renyi entropy 
power, coincides with the range for which there is mass 
conservation for the solution of (O J4). 

The rest of this paper is devoted to the proof of (|8). Before 
entering into technical details, we will however explain in 
some details the physical reasons which in our opinion suggest 
to define the p-th Renyi entropy power in the form (0. To this 
aim, we will introduce in Section [TT] some known facts about 
nonlinear heat equations (cfr. the book by Vazquez l2D . which 
fully treats the case p > 1, and [4 | for the case p < 1). Among 
other facts, this connection between Renyi entropies and the 
nonlinear heat equations allows to recover in a simple way 
a related p-th Fisher information recently considered in fl4| . 



(6) ma. 



II. Self-similar solutions and Renyi entropies 

The relationship between Shannon's entropy power and 
the solution to the linear heat equation (0 can be fruitfully 
highlighted owing to the fundamental solution, representing 
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heat release from a point source (here the origin x — without 
loss of generality). The source-type solution of unit mass at 
time t > is represented by the Gaussian density 

of variance equal to 2n (8). Since the Shannon's entropy of 
M(-,t) equals 

71 

H(M(-,f)) = -]og(47ret), 

it follows that the corresponding entropy power is a linear 
function of time, i.e. 

d 2 

N(Af (•,*)) = Airet, hence _N(M(*)) = 0. 

In view of this remark, the concavity property of entropy 
power can be rephrased by saying that for all times t > 
the fundamental source-type solution maximizes the second 
derivative of the Shannon's entropy power among all possible 
solutions to the heat equation. 

This idea easily extends to the nonlinear heat equation (0. 
The corresponding fundamental solution was found around 
1950 by Zel'dovich and Kompaneets and Barenblatt ||2T1 . In 
the case p > 1 (see |4] for p < 1) the Barenblatt (also called 
self-similar or generalized Gaussian) solution departing from 
x = takes the self-similar form (recall the definition of /z in 
©) 

KM--=^M p (JL), (io) 

where 

M p (x) = (C- K \x\ 2 )f*; (11) 

here (s)+ = max{s,0}, k := ^— , and the constant C 
in ( fTTT ) can be chosen to fix the mass of the source-type 
Barenblatt solution equal to one. 

Notice that, if we consider the mass-preserving rescaling 

■R a : f(x) -> KJ(x) := a~ n f (x/a) , a > 0, x E M", 

of a given nonnegative density /, a direct computation imme- 
diately yields 

R p (K a f) = Hp(/) +nloga, N p (3? a /) = a"N p (/). (12) 

An application of dT2b to ( TTOb with a := then shows that 
the p-th Renyi entropy power defined in ([3]) is a linear function 
of time 

N p (M p (t)) = N p (M p )t, so that -^N p (M p (t)) = 0. 

As before, the concavity property of Renyi entropy power 
would imply that for all times f > the Barenblatt source-type 
solution maximizes the second derivative of the p-th Renyi 
entropy power, among all possible solutions to the nonlinear 
heat equation. 



III. A new Lyapunov functional 

The proof of (O requires to evaluate two time derivatives 
of the p-th Renyi entropy power, along the solution to the 
nonlinear heat equation (0. The first derivative of the entropy 
power can be easily obtained: in fact, if u(-,t) solves the 
nonlinear heat equation (O, integration by parts immediately 
leads to (cfr. Appendix 151 

^H p «,t)) =I p ( u (.,i)), i>0, (13) 

where 

P (X) = I p (/) := - — — - / — — — dx. (14) 

When p 1, identity ( fT3T > reduces to DeBruijn's identity, 
which connects Shannon's entropy functional with the Fisher 
information of a random variable with density 

0{X) =!(/):= f ^j*fdx, (15) 

via the heat equation. Analogous computations can be done 
here. Using identity ( fT3l we get 

— N p (u(-,t))=i/ N p («(-, <))!,(«(•, t)). 

Let us introduce the quantity 

T„(u) := N p (u) l p {u), v(t) := ? p (u(;t)). (16) 

Since 2/n + p — 1 > 0, the concavity of entropy power 
can be rephrased as the decreasing in time property of t >-} 

T p (u(;t)). 

It is important to notice that by (fl2l i and the scaling property 
of the p-th Fisher information 

l P (3iaf) = a-' 1 I p (f), (17) 

the functional T p is invariant under the dilations 3£., i.e. 

T P (Xaf) = Tp(f) for every a > 0. (18) 

Property ( fT8l allows to identify the long-time behavior of the 
function v(t) = T p (u(-,t)). It is nonincreasing, and it will 
reach its infimum as time t — y oo. The computation of the 
limit value uses in a substantial way the scaling invariance 
property. In fact, we can rescale u(x, t) according to 

U(x,t) = t~ n/f "u(xt~ 1/f ",t) = %^„u{-,t) (19) 

where fi is defined in (O as usual, so that 

v(t) = T p (u(-,t)) = T p (U(-,t)) for every t > 0. 

On the other hand, it is well-known that (cfr. for example |[2D ) 

lim U(x,t) =MJx), (20) 

t— too 

the Barenblatt profile defined in (JTTJ. Therefore, denoting by 
f(x) := u(x, 0) the initial probability density, passing to the 
limit one obtains the (isoperimetric) inequality for the p-th 
Renyi entropy: 

Theorem 2: If p > nj (n + 2) every smooth, strictly positive 
and rapidly decaying probability density / satisfies 

T„(/) - N p (f) I p (f) > N P (M P ) l p (M p ) =: 7n , p , (21) 
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where the value of the strictly positive constant j n , p is given V. PROOF OF THE CONCAVITY OF RENYI ENTROPY POWER 
m and <@1 of Appendix S Arguing afj ^ m k [$ sufficient to consider the case 



of smooth, strictly positive and rapidly decaying probability 

IV. SOBOLEV INEQUALITY REVISITED densities 

Inequality (EB can be rewritten in a form more suitable to For a given probability density u we set 
functional analysis. Let f(x) be a probability density in W 1 . ^ 
Then, if p > n/(n + 2) EJu) := e p (u(x))dx, where e„(r) := 



2 + 2re(p-l) 

f \^f p ( x )\ ^ x > | /" jp^'j da ,] * <P 11 ^22) Consequently, the p-th Renyi entropy of u can be written as 

7r« /(&) ' \7r" / l / \ 

If n > 2, the case p = (n - l)/n is distinguished from the H p (u) := — — log (^(p - l)E p (u) J . 
others, since it leads to 

„ . . Likewise, since e'(it) = quP^ 1 , q := -^y, we can write 

2 + 2n(p — 1) g 1 ^ p 1 



n(p- 1) n' /■ |D u p| 2 



Ni-i/n(/) = J 1 - 1/n (x)dx. = / | De ;( u )| 2 u dx 



and 



In this case the concavity of N]_iy n along (0 has been already R eca jj a j so m at 
known and has a nice geometric interpretation in terms of 

transport distances, see fl5l . = u p = ue'Ju) — e p (u), L'Ju) = ue''(u). 

Note that the restriction n > 2 implies (n— l)/n > n/(n + 
2). Hence, for p = (n - l)/n we obtain that the probability The nonlinear heat equation @ can be equivalently written as 

density f satisfies the inequality „ _ / „ ; , >\ „ 

^ -l ^ _ V ■ I uDe (u) J = 0. (26) 

, | V/ (n-l)/n (a;) |2 V / 

/ ri \ ax — 7n,(n-i)/n- Using equation ( I26l i. and integrating by parts, we obtain (see 

JR™ J\ x ) . ,. 

Appendix B) 

The substitution / = g 2 , where 2* = 2n/(n — 2), yields d 

- — E p (u t ) = Fp(u), (27) 

| V/ (n-l)/n (:c) |2 / 2 n-2\ 2 f .„ , , |2j d * 

;l -d.T=( / |V«7(x)| 2 dx. 

d 



/(x) V » - 2 

Therefore, for any given function g > such that g(x) 2 is a dt 

probability density in R", with n > 2, we obtain the inequality _ 2 f u pf\j) 2 e ' ( M )| 2 + (p _ l)(Ae' (u)) 2 ) da; 

/" / - 2 \ 2 P v ' 

\ \Vg(x)\ 2 dx>(- 

Tn,(n— l)/n* 

(24) 

For a given function <p t which depends on time, and a positive 

A careful computation (see Appendix A) gives 



D p (u t ) := - -F p {u t ) (28) 



constant a 



2 2 (n-l) 2 /r(n/2)\ 2/ " ^tj exp(cr t ) = exp^t) + (ct^) 2 ) . 

7n,(n-l)/n = "~ 



n-2 V r(n) / Therefore 



and simple scaling argument finally shows that, if g(x) has d 2 nil 

a mass different from 1, g satisfies the Sobolev inequality 0], ^2 ex P(°W - _cr ^t - ( CT ^t) • 

^ If (j> t := H p (u t ), where u t = u(-,t), we have 

f \Vg(x)\ 2 dx>$ n (j g(x) r dx\ , (25) 1 -F p (u t ) 



where 



(29) 



p - 1 Ep(ut) 

r(n/2)\ 2/ ™ and ^ ^ m=» /.^ t, ^2 



§„ = n(n - 2)tt ( - I „ = 1 Dp(u t ) E p (u t ) - F p (M t; ^ 

P-l Ep(u(t)) 9 



is the sharp Sobolev constant. Hence, Sobolev inequality with 

the sharp constant is a consequence of the concavity of Renyi Hence we end up with the condition 

entropy power of parameter p = (n - l)/n, when n > 2. g / _ 2 \ / g \ 2 

In all the other cases, the concavity of Renyi entropy ^ _ 1 [ u P ( u t) F p {u t ) b p (u t ) J > ^ J I'piwtJ , 
power leads to Gagliardo-Nirenberg type inequalities with 

sharp constants, like the ones recently studied by Del Pino and l e ' ( su PP ressm g the index t) 

Dolbeault [9], and Cordero-Erausquin, Nazaret, and Villani, f , „ . 2 

with different methods. aD P^ J dx * ( a + a[ V ~ X ))?v^) ■ 
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Since a > 0, the second derivative is non positive if 

D p (u) / u?da;> (a + (p-l)){F p (u)) 2 . 



one obtains 



(31) 
Since 

F p (u) = J Bu p ■ De p (u) dx = - J u p Ae' p {u) dx, 
by Cauchy-Schwarz inequality we have 

F P (u) 2 < j vPdx J u p (Ae' p (u)) 2 dx. 
It follows that ([3]} holds if 

D p («) > (a + (p - 1)) / ^(Ae^H) 2 da. 

On the other hand, by the trace inequality, |D 2 /| 2 > i(A/) 2 



D» > 2 (- + (p - 1)) / u p (Ae;( w ))^ dx, 

and we end up with the condition 

2 

er < hp — \ = v. 

n 

Choosing for a the upper bound in d32l we conclude. 



(32) 



Appendix A 
Computation of the constants 

Let us recall here some useful formulas. The surface of the 
n — 1 dimensional unit sphere §™ _1 is given by = 
27r n / 2 /r(n/2). Let us first consider the case p > 1. If a > 0, 
using the integral representation of Beta function we have 



(i-\ x \'y + dx = \s n -'\ / p"- 1 (i-pT^ = 

Jo 

o_n/2 /-l o n/2 

t n/2-l (1 _ t) a (ft= ^L^ B( 2 ja+1 ) 



r(n/2) y 

„»/2 r (« + !) 

TV 1 



T(n/2) y 2' 



r(f + a + l) 



With this formula, we can evaluate quantities associated to the 
Barenblatt function ( fTTT i. Indeed, if 



(1 - \x\ 2 ) 1 ' [p - 1) dx = W 2 



r p±i 



r ( | + £±i 



, (33) 



we obtain a Barenblatt of mass equal to one, we denote by 

%{ X ) = (c p - M 2 )^- 1} 

if 

2(p-l) 

C p = A p " <3I - 1)+2 . (34) 

Also lF2ll 



\x\ 2 %{x)dx= - 1} C p , 

(n + 2)p — n 



(35) 



and, since 



■B p (x) p dx= / (C p - |xr)'B p (a;)dx, 



£p(x) p da; 



2p 



-C B 



(36) 



(n + 2)p — n p 

Thanks to ( l35l l and ( l36l l. we reckon the values of the p-Fisher 
information defined in (fl4l i and of the Renyi entropy 

Hp(/), associated to 23p 



and 



Hp(a3p) = -^lo gr — ^ 

1 — p (n + 2)p — n 

2n 

Ip(Sp)=n-^ T . 



Cp, (37) 
(38) 



Hence, if p > 1 the value of the constant j n<p is 

2p 



7n, P = nw 



p-1 




2 + Ti(p-l) 

(n + 2)p - n\ "(p- 1 ) 
2p 



(39) 



Analogous computations can be done in the case p < 1. In 
this case 



r 



A p = j (1 + \x\ 2 ) l ^ p -^ dx = tt" /z v '/, , (40) 



n/2 U-P 2 



while 



|x| 2 Sp(x) dx = P) Cp. (41) 

(n + 2jp — n 



Note that, if p < 1, the second moment of the Barenblatt 
is bounded if and only if p > n/{n + 2). Therefore, the 
computations that follow are restricted to this domain of p. 
If this is the case, formula d36t still holds, while 



Ip(Sp) = n 

Finally, if n/(n + 2) < p < 1, 
2p 



2p 
1-p 



(42) 



ln, P = nir 



1-p 




(43) 



Appendix B 
Proof of identities d27j» and ( f28b 

d2~7b follows by a simple integration by parts: 

^E p (u t ) = Je p (u)d t udx = J e' p (u) V • (u De' p (ufj da; 
= -/D< W .De; W „d,. 

( T28] > is based on the Bochner identity or, equivalently, Bakry- 
Emery T-calculus (in their simplest Euclidean form), see ifTBI 
for analogous computations of the second derivative of E p 
along geodesies in the Wasserstein space. 
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One has 

d t e' p (u) = e';{u)d t u = e£(u) V • (uDe' p (u) 

= ue£(u) Ae' p (u) + e' p \u) D« • De' p (u) 
= L' p (u)Ae' p (u) + \Be / p (u)\ 2 . 
Recall the Bochner identity 

2T 2 (g) = A\Bg\ 2 - 2Bg ■ DA.g = 2|D 2 5 | 2 , 

and 

(rL'<r)-Ly = rL''(r), 



while 



(rL' p (r)-L)'e';(r) = L';(r)L' p (r). 



Then 
(1 



F P (u t ) = J (\Be' p (u)\ 2 d t u + 2uDe' p {u)-'Dd t e' p (u))dx 
= J \De' p {u)\ 2 AuP dx 

+ 2 J u~De' p (u)-D(L p (u) Ae' p (u) + \Ve p (u)\ 2 ^) dx 
= J u p A\De' p {u)\ 2 dx + 2 J uL' p {u) Be' p (u) • BAe' p {u) dx 

+ 2 J BL p {u) ■ BL p (u) Ae' p (u) dx - 2 J L p (u)A\Be' p (u) \ 2 
= - J L p {u)A\De' p {u)\ 2 dx + 2 J L p {u) Be' p (u) ■ DAe' p (u) dx 
+ 2 J (uL' p (u) - L p (u)) Be' p (u) ■ BAe' p (u) dx 
+ 2 J BL p (u) • BL' p (u) Ae' p (u) dx 
= - J L p (u)T 2 (e' p (u))dx 

-2 J (uL' p (u) - L v {u)) {Ae' p (u)) 2 dx 
= -2 J L p (u)\B 2 e' p (u)\ 2 + (uL' p (u) - L p (u)) (Ae») 2 dx 
= -2 I u»(\D 2 e' p (u)\ 2 + (p-l)(Ae' p (u)) 2 )dx. 
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